Single pass graph sparsification in distributed stream processing

We give a distributed one pass streaming algorithm for graph sparsification. Besides producing a sparsifier, our algorithm maintains a hierarchy of UNION-FIND data structures in a distributed manner that efficiently support queries of strong connectivities between pairs of vertices. An important component of the algorithm is an implementation of UNION-FIND queries over an Active Distributed Hash Table that guarantees good load balancing properties. This is achieved via a single step of what is known in the literature as the zig-zag heuristic. We provide theoretical guarantees for the load balancing achieved by this heuristic, and show how the structure of our sparsification scheme ensures good load balancing across the hierarchy of UNION-FIND data structures maintained by the algorithm. We also present simulation results on synthetic as well as real world data verifying the load balancing properties and the quality of approximation of strong connectivities achieved by the algorithm.